When the Boards Fail Us-Structural Bias in Certification Examinations.

This essay shines a light on structural bias inherent to the board certification examination process, sharing the author's experience preparing and sitting for the examination while contending with co-occurring challenging life events.

A method to quantify and compare clinicians' assessments of patient understanding during counseling of standardized patients.

OBJECTIVE: To introduce a method for quantifying clinicians' use of assessment of understanding (AU) questions, and to examine medicine residents' AU usage during counseling of standardized patients about prostate or breast cancer screening. METHODS: Explicit-criteria abstraction was done on 86 transcripts, using a data dictionary for 4 AU types. We also developed a procedure for estimating the "load" of informational content for which the clinician has not yet assessed understanding. RESULTS: Duplicate abstraction revealed reliability kappa=0.96. Definite criteria for at least one AU were found in 68/86 transcripts (79%). Of these, 2 transcripts contained a request for a teach-back ("what is your understanding of this?"), 2 contained an open-ended AU, 46 (54%) contained only a close-ended AU, and 18 (21%) only contained an "OK?" question. The load calculation identified long stretches of conversation without an AU. CONCLUSION: Many residents' transcripts lacked AUs, and included AUs were often ineffectively phrased or inefficiently timed. Many patients may not understand clinicians, and many clinicians may be unaware of patients' confusion. PRACTICE IMPLICATIONS: Effective AU usage is important enough to be encouraged by training programs and targeted by population-scale quality improvement programs. This quantitative method should be useful in population-scale measurement of AU usage.

Assessment of parental understanding by pediatric residents during counseling after newborn genetic screening.

OBJECTIVE: To investigate pediatric residents' efforts to assess understanding in discussions about positive newborn screening test results. Newborn screening saves lives, but confusion about false-positive and carrier results often leads to psychosocial problems. DESIGN: Explicit-criteria abstraction of transcripts of encounters with standardized parents of a fictitious infant found to carry cystic fibrosis or sickle cell hemoglobinopathy. SETTING: Simulated doctor-patient encounter. PARTICIPANTS: Pediatric residents participating in an educational workshop on how to inform parents about positive newborn screening test results. MAIN OUTCOME MEASURES: Abstraction used an explicit-criteria data dictionary with definitions for 5 different ways to assess understanding. A "partial" designation was used for leading syntax or no pause for response. RESULTS: Interabstractor reliability over 59 transcripts (2 per resident) was kappa = 0.93. Only 26 of 59 transcripts (44.1%) met definite criteria for at least 1 assessment of understanding. Most assessments were the less effective close-ended (37.3% of transcripts) and "OK?" question types (32.2% of transcripts). Only 3 transcripts met definite criteria for an open-ended assessment and no transcripts included a request for a teach-back, the type thought to be most effective. Four transcripts (6.8%) included an advance request for questions. With partial-criteria assessments included, an additional 31 transcripts (52%) were identified. CONCLUSIONS: The small number of assessments of understanding and the high fraction of less effective assessments do not bode well for parental understanding, especially for parents with limited health literacy. Training programs should address assessments of understanding, but quality improvement activities using these types of assessment methods may also be needed.